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Abstract 

Researchers are often faced with the dilemma of having to use intact groups. ANCOVA 
is sometimes employed as a means of leveling unequal groups. However, only after five 
assumptions are met is it safe to use ANCOVA. Part and partial correlations also invoke 
statistical correction procedures. The purpose of the present paper is to (a) review these 
covariance correction procedures, (b) explain the relevant assumptions and potential pitfalls 
related to the analyses, and (c) discuss research situations when their use is both warranted and 
ill-advised. 
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Covariance Corrections: What They Are and What They Are Not 

Researchers often find themselves at the mercy of intact groups (e.g., classrooms, 
schools, etc.) that limit, if not eliminate, the possibility of conducting true, randomized 
experimental research (Henson, 1998). For example, given a particular intervention for special 
education instruction, it is arguably unethical to randomly assign students into treatment and 
control group participants. Accordingly, there remains a great need for experimental validation 
of educational programs in general (cf. Welch & Walberg, 1974). 

One result of this dynamic is the tendency (perhaps necessity) to either (a) use existing 
groups (e.g., two special education classrooms) and attempt to statistically control for preexisting 
differences between groups or (b) use one treatment group and attempt to statistically control for 
preexisting variables that may impact the final outcome(s). These types of analyses are all 
related and go by various names such as analysis of covariance (ANCOVA) and part and partial 
correlation. 

Covariance corrections vary in their frequency of use. ANCOVA, for example, appeared 
in about 4% of the published literature in American Educational Research Journal articles 
(Goodwin & Goodwin, 1985). More recently, Baumberg and Bangert (1996) reported that the 
use of ANCOVA accounted for 7% of the statistics used in the research found in the Journal of 
Learning Disabilities for 1989 through 1993. Similarly, in a review of 17 well-respected 
education and psychology journals, Keselman and colleagues (1998) found that univariate 
ANCOVA was used in 7 % of the articles examined (45 out of 65 1). Furthermore, this hit rate 
does not include applications of regression analyses with covariates or multivariate ANCOVA. 
Thompson (1988, 1994) found frequent use among doctoral students. Furthermore, it is arguable 
that other related corrections, such as part and partial correlation, are more frequently utilized in 
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the literature, particularly when researchers attempt to control for variables that may influence 
the outcome of a single treatment group, such as using a pretest correction on posttest scores. 

Like most analytic methods, it is important to realize that these corrections have certain 
benefits and limitations. ANCOVA, for example, is generally inappropriate when used with 
intact groups such as established classrooms of students. And yet, ANCOVA does continue to 
be utilized in such situations. Of the 45 uses of ANCOVA reported, by Keselman and colleagues 
(1998), full two-thirds of the studies "involved non randomization of the experimental units" (p. 
373). As suggest by Henson (1998), 

Perhaps the lingering use of ANCOVA is due to the mystical promise that ANCOVA is a 
statistical correction of all pre-treatment problems and that it will provide increased 
power against Type II error. Such an argument is particularly compelling to doctoral 
students who find themselves aggressively seeking and even praying for statistically 
significant results! Unfortunately, ANCOVA has multiple assumptions that must be met 
before it can be accurately utilized, (p. 4) 

Similarly, part and partial correlations should be used only when it is meaningful to do so. 

While these correlational procedures may not have the same assumption as ANCOVA, there are 
important ramifications of their misuse. 

The purpose of the present paper is to (a) review these covariance correction procedures, 
(b) explain the relevant assumptions and potential pitfalls related to the analyses, and (c) discuss 
research situations when their use is both warranted and ill-advised. To make the discussion 
concrete, heuristic data sets are presented and analyzed to illustrate appropriate and inappropriate 
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A Review of Procedures 

ANCOVA is a form of statistical control used to determine whether some variable, other 
than the intervention, is responsible for the difference on the dependent variable between groups 
(Borg & Gall, 1989; Shavelson, 1996). To do so, a continuously scaled variable that is highly 
correlated with the outcome measure is identified that contains information on individual 
subjects and is collected before the intervention begins. This variable is called the covariate. 
Covariates may take many forms and are best determined by the theoretical model of the 
researcher. Examples of covariates in the social and behavioral sciences may include a pretest, 
aptitude, age, experience, previous training, motivation, and grade point average (Ruiz-Primo, 
Mitchell, & Shavelson, 1996). Importantly, ANCOVA is designed to be used when individuals 
have been randomly assigned to control and experimental groups. That is, the ANCOVA 
statistical analysis is appropriate when the study has an experimental design . It cannot make a 
study’s design experimental when, in fact, it is not. As Henson (1998) noted. 

In a researcher’s zealous attempts to have an experimental or quasi-experimental design, 
the use of ANCOVA comes to be seen as a way of making such a design happen. In fact, 
a design is either experimental or it is not. Statistical analyses employed on the data 
obtained from the design do not magically transform a study into a true experiment . . . 
This thinking (or lack thereof) reflects confusion regarding the cooperative but separate 
roles of methodological design and statistical analysis, (p. 6-7) 

In an ANCOVA, covariate regression first removes the covariate variance from the 
dependent variable ignoring treatment group membership, thus reducing the error variance from 
within groups or the error caused by systematic differences among subjects. The idea is that this 
gives a more accurate estimate of error when asking if two or more groups differ on a dependent 
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variable. If the covariate and the dependent variable are linearly related, the use of the covariate 
in the analysis will serve to reduce the unexplained variation between groups. The greater the 
magnitude of the correlation, the greater the reduction in dependent variable variance due to 
within groups variation. Next, the error (or residualized) scores that remain from this regression 
of the covariate and dependent variable become the new dependent variable, and an analysis of 
variance (ANOVA) is performed using this residualized dependent variable (Henson, 1998). 
Since removal of the covariate ideally reduces the variance of the error scores in this ANOVA 
analysis, the size of the ANOVA F statistic purportedly increases, as does the likelihood of 
obtaining statistical significance. If error variance is reduced in this manner (which assumes that 
certain assumptions have been met), the ANCOVA will be more powerful than the ANOVA 
(Shavelson, 1996). 

In sum, ANCOVA partitions the total variability among the scores on the dependent 
variable into three sources of variability (Ruiz-Primo et al., 1996) but does so in a two step 
sequence. First, variance attributable to the covariate is removed from the dependent variable. 
This is done for all subjects, regardless of treatment group membership. Second, a classical 
ANOVA partitions the residualized dependent variable into variance attributable to the treatment 
group (or explained) variance and error. 

As previously stated, one reason ANCOVA is used concerns its “promise” of power 
against Type II error. However, there are several conditions that must be met for this to occur. 
Statistically, ANCOVA requires one degree of freedom be lost from the error term for each 
covariate used (Shavelson, 1996). This actually results in a larger critical value for the F statistic 
and makes it more difficult to find a significant F statistic. In order for power to increase, the 
variance attributable to the covariate must be sufficient enough to compensate for the loss of 
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power due to degree of freedom manipulations. This compensation occurs by lowering the sum 
of squares error (because some of the variance was due to the covariate) and thereby increasing 
power in the ANOVA. Thus, power is fostered only when the covariate has a sufficiently large 
correlation with the dependent variable. If the covariate is not highly correlated with the 
dependent variable, the use of ANCOVA may actually lower statistical power. 

A similar procedure of covariance correction is employed in part and partial correlations, 
which are used to identify unique relationships between two variables. Part correlation removes 
the influence of a third variable from one of two variables being considered (Borg & Gall, 1989). 
Here, a regression is performed between the third variable and one of the two variables being 
considered. The remaining error scores (residuals) are then correlated with the second variable 
in consideration, yielding the part correlation. Theoretically, the analysis is intended to explain 
the unique relationship between the two variables of interest after removing the spurious 
influence of some identified third variable. Partial correlation involves a similar dynamic but 
removes the influence of a third variable from both of the variables being considered. Here, two 
regression analyses are conducted between a third variable and both variables in consideration. 
Then, the remaining error scores from both of these analyses are correlated, yielding the partial 
correlation. Theoretically, a partial correlation explains the relationship between the variables 
after removing the spurious effect of the third variable from both variables of interest. 

For example, in considering factors that influence end-of-the-year test scores, one would 
have to consider what the student knew coming into the class (beginning-of-the-year test scores) 
as well as the teaching method that was used. Naturally, the beginning-of-the-year test scores 
would influence the teacher's decision of which method of teaching to use. Teachers with high 
achievers will probably instruct them differently than teachers of low achievers. By using partial 
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correlation analysis, the influence of the beginning-of-the-year achievement level can be 
removed from the end-of-the-year scores and from teacher instructional factors. This same 
scenario could employ part correlation analysis to remove the beginning-of-the-year achievement 
from just one variable, such as the end-of-the-year scores, to purportedly account for differences 
between students prior to instruction. In this case, if a large correlation were then observed 
between teaching method and the residualized end-of-the-year scores, the researchers may be 
more inclined to believe that the instructional method had some real effect on student 
achievement. (Note that this assertion could only ultimately be supported through an 
experimental study.) However, in such pre/posttest cases, there are serious conceptual problems 
with using part and partial correlations that will be discussed later. 

Relevant Assumptions and Potential Pitfalls 

If ANCOVA is to be used appropriately, then multiple conditions must be satisfied. 
Misinterpretation and misapplication of ANCOVA data may result from ignoring required 
assumptions. Researchers, both novice and experienced, should heed warnings against using this 
analysis as a “correction” of the dependent variable scores to adjust for data set problems (Buser, 
1995), particularly when treatment groups were not randomly assigned. Loftin and Madison 
(1991) listed several assumptions underlying the successful use of ANCOVA. They are as 
follows: 

1 . The covariate (or covariates) should be an independent variable highly correlated 
with the dependent variable. 

2. The covariate should be uncorrelated with the independent variable or variables. 

3. With respect to the dependent variable, (a) the residualized dependent variable (Y*) is 
assumed to be normally distributed for each level of the independent variable, and (b) 
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the variances of the residualized dependent variable (Y*) for each level of the 
independent variable are assumed to be equal. 

4. The covariate and the dependent variable must have a linear relationship, at least in 
conventional ANCOVA analyses. 

5. The regression slopes between the co variate and the dependent variable must be 
parallel for each independent variable group, (p. 134) 

Each assumption has important ramifications. For example. Assumption 1 states the need 
for high correlation between the dependent variable (Y) and the covariate (X). The covariate 
will not do much to reduce the error sum of squares if the correlation is not high, and power may 
actually be reduced due to the loss of error degree(s) of freedom as the price to use the 
covariate(s). To demonstrate this point. Table 1 represents three sets of summary tables 
illustrating a regular ANOVA without a covariate, an ANCOVA in which the covariate (X) has a 
small correlation vrith the dependent variable (Y), and an ANCOVA involving a covariate with a 
large correlation with the dependent variable. The covariate is perfectly uncorrelated with the 
treatment conditions. 



INSERT TABLE 1 ABOUT HERE. 



Notice that when the covariate has a large correlation with the dependent variable, the 
statistical adjustments include the loss of one degree of freedom (df) for each covariate, a larger 
mean square (MS) error, and larger calculated F values. While the lowering of degrees of 
freedom error actually increases the critical F, the decrease in error sum of squares 
counterbalances the loss of degrees of freedom in error due to the high correlation between the 
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covariate and the dependent variable. If the covariate is perfectly uncorrelated with the 
dependent variable, the sum of squares error would remain totally unchanged after the covariate 
is removed. In addition, degrees of freedom error would decrease by one for each of the 
covariates, mean square error would increase, and F calculated would become smaller. All of 
these results make it less likely the researcher will get statistical significance. 

Assumption 2 obligates the covariates and the independent variables to be uncorrelated. 
Table 2 presents three summary tables to illustrate the importance of this assumption. The best 
situation is illustrated in the (a) portion of Table 2, where the covariate (X) and the independent 
variable are perfectly uncorrelated. This allows unique explanation of different portions of the 
total variance of the dependent variable by the covariate and independent variable. However, 
reality is such that most social science variables are correlated to some degree. When the 
covariate and independent variable are indeed related, it is very possible that the covariate will 
“rob” the treatment effect of explanatory power. 

In part (b) of Table 2, the covariate and the treatment variable are moderately correlated. 
Since a covariance regression is conducted in ANCOVA prior to the ANOVA analysis, some of 
the sum of squares that could have been attributed to the treatment variable were instead 
accounted for by the covariate. In fact, all of the shared variance between the covariate and the 
treatment will be attributed to the covariate, potentially “robbing” the treatment variable of 
dependent variable variance. 

Finally, part (c) of Table 2 illustrates the extreme ramifications of this problem. Here, all 
of the sums of squares of the treatment are accounted for by the covariate, and no credit is given 
to the treatment variable. The unsuspecting researcher may conclude that the treatment had no 
effect whatsoever. 



O 

ERIC 



11 ' 



Covariance Corrections 1 1 



INSERT TABLE 2 ABOUT HERE. 



This problem is not unlikely. As noted, most social science variables are correlated to 
some degree. Furthermore, when treatment groups are not randomly assigned, then there is an 
increased likelihood that the covariate will be correlated with the treatment groups. For example, 
if a researcher wished to study achievement of students in two special education classrooms, he 
or she may choose to use some pretest measure of achievement as a covariate to “level” any 
differences between the classes since they were not randomly assigned. It is very possible that 
the pretest achievement scores (covariate) will be related to what classroom the students attend. 

In this case, it may be ill advised to use that covariate since it would remove some of the 
variance from the treatment variable, thereby lowering observed effect of the treatment in the 
ANCOVA analysis. Thompson (1994) explained this quandary; 

When the covariate is related to the treatment variable, use of the covariance correction 
will alter the effects attributed to the treatment itself. For example, one might have a very 
effective intervention that looks completely ineffectual, because the covariate is given 
credit for the variance that would correctly otherwise be attributed to the treatment 
variable. Here the ANCOVA correction actually destroys power against Type II error. 

(p. 27) 

Paradoxically, when groups are not randomly assigned, ANCOVA is needed the most to 
“level” for group differences; but in these cases, ANCOVA is less appropriate as an analysis 
when groups are significantly different. This is a paradox that severely limits the usefulness of 
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ANCOVA outside of studies that use random assignment, which, of course, are rare in 
educational research. 

In the third assumption, the residualized dependent variable must meet the requirements 
of any ANOVA analysis. Since the residualized dependent variable becomes the dependent 
variable in an ANOVA, it must be normally distributed and meet the homogeneity of variance 
assumption. Homogeneity of variance means that the variance of the residualized dependent 
variable is the same for all levels of the independent variable. It is only reasonable to compare 
the means of each group when the groups’ distributions are equally variable. 

Assumption 4 addresses the regression analysis assumption that a linear relationship 
exists between the covariate and the dependent variable. If the covariate is not linearly related to 
the dependent variable, use of conventional ANCOVA may bias the data. To meet this 
assumption, change in the covariate should correspond to a proportional change in the dependent 
variable for each level of the independent variable. It is important that the proportional change 
exist for all levels of the independent variable to avoid biasing the responses from one or more of 
the groups (Loftin & Madison, 1991). 

Finally, assumption 5 demands that an ANCOVA analysis possess what is known as 
homogeneity of regression. This assumption states that "The regression slopes between the 
covariate and the dependent variable must be parallel [or homogeneous] for each independent 
variable group" (Loftin & Madison, 1991, p.l34). If the slopes are not parallel, then one line 
caimot represent both the covariate and dependent variable accurately (Henson, 1998). In this 
case, the data will be biased rather than corrected since the relationship of the independent 
variable to the dependent variable will be changed (Loftin & Madison, 1991). It is important to 
remember here that the initial regression between the covariate and the dependent variable is 
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conducted ignoring group membership (as defined by the independent variable). Therefore, a 
“pooled” regression line is used to represent the regression slopes between the covariate and the 
dependent variable for each individual group. This is only appropriate when the “pooled” 
regression line accurately represents (or has the same slope of) the individual group regression 
lines. 

To illustrate this assumption, an ANCOVA analysis will be conducted on a small 
heuristic data set taken from Henson (1998). The SPSS syntax used to run the analysis is 
included in the Appendix. In this example, a researcher is investigating whether a note-taking 
intervention will positively impact achievement of special education students. Unfortunately, the 
researcher is compelled by the school’s administration to provide the intervention to all special 
education classrooms, thereby leaving no possibility for a true control (or even a comparison) 
group. Rather than having no comparison group, the researcher decides to use a regular 
education class as a comparison group and statistically “equate” the groups with ANCOVA 
using a reading and writing achievement measure as a covariate on the achievement dependent 
variable. Table 3 presents the data for the treatment (special education) and comparison (regular 
education) groups. Scores are given as T-scores. The results of the ANCOVA analysis are given 
in Table 4. 



INSERT TABLES 3-4 ABOUT HERE. 



In this example, when the regression between the covariate (READ) and dependent 
variable (ACHIEVE) was conducted, a “pooled” regression equation was derived that ignored 
group membership (Y*=-9.158192 + 1.405085 * READ). In other words, the relationship 
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between READ and ACHIEVE was examined simultaneously for all 12 students. However, it is 
intuitive to note in this case that these 12 students are not the same. As would be expected, the 
group means on the covariate variable are notably different (Treatment group: M = 38.33, SD = 
6.83; Comparison group: M = 56.67, SD = 6.83). In this case, the covariate variable is strongly 
related to the treatment variable (see Assumption 2 above). 

Furthermore, the use of the “pooled” regression line to represent all groups is appropriate 
only when it accurately represents the regression lines between the covariate and the dependent 
variable for each individual group (i.e., when the lines are parallel). In this case, the “pooled” 
regression line does not represent either of the individual group regression slopes. Figure 1 
presents the “pooled” slope and the individual group slopes. Thompson (1988) presents the 
researcher with the "nasty dilemma" of ANCOVA: 

If the controls are not needed, then they should not be used. But if statistical control is 
needed because the groups in a study are not equivalent, then often the homogeneity of 
regression assumption cannot be met and the use results in seriously distorted inferences, 
(p. 23) 



INSERT FIGURE 1 ABOUT HERE. 



As previously noted, part and partial correlations are close cousins to the ANCOVA 
procedure, and all are part of a statistical correction family that attempts to control for influences 
on variables. While part and partial correlation do not have the same assumptions of ANCOVA, 
they do require that the general assumptions of linear regression and correlation analysis be met 
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since both are hybrids of a regression analysis followed by a correlation analysis with error 
scores. 

Importantly, in all statistical correction procedures, one must be able to interpret the 
residualized variance after the covariance correction has occurred. In the above ANCOVA 
example, one is compelled to question what is left of the dependent variable after the covariate 
had a 96.01% effect size (i.e., the covariate accounted for nearly all of the dependent variable 
variance). Of the variance that remained, was it still achievement as originally expected, or was 
the dependent variable some conglomeration of error and other constructs? These conceptual 
difficulties of using covariance corrections should not be overlooked. As Thompson (1992) 
noted, "Statistical corrections remove parts of the dependent variable and then analyze 
whatever's left, even if whatever's left no longer makes any sense. At some point we may no 
longer know what it is we're analyzing" (p. xiii-xiv). 

For example, it is not uncommon to find researchers employing some form of part or 
partial correlation when using pre and posttest measures. Most commonly, a pretest is given as a 
baseline measure and used as a covariate on the dependent variable posttest. Then, a third 
variable may be correlated with the residualized posttest variance (part correlation). The intent 
of the procedure is to control for differences between subjects prior to the intervention and then 
analyze differences that occurred during the time between pre and posttest. 

While the logic of the analysis is appealing, it is conceptually troublesome to use a pretest 
as a covariate for a posttest measure. Interpretation of the dependent variable becomes a tricky 
matter when a researcher has removed variance due to the construct at pretest. To illustrate this 
dynamic, consider the following heuristic example. Suppose a researcher wishes to study the 
effects of collaboration during an academic year-long teacher inservice on teachers' sense of self- 
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efficacy. The same pre and posttest measure of efficacy is given prior to and after the inservice. 
The researcher then removes pretest efficacy variance from posttest efficacy (the covariance 
correction) and then correlates the residualized dependent variable (posttest efficacy error scores) 
with a collaboration variable. 

As an example, Table 5 presents two sets of hypothetical data for five teachers. In the 
first set of data, all of the teachers gained exactly the same during the inservice with all teachers 
increasing by one point on a Likert scale from pre to posttest. This may be considered a best 
case scenario in that all teachers benefited equally from the inservice. Unfortunately, when the 
pretest scores are covaried with the posttest scores, there is exactly zero residualized variance 
that remains. This is because the pre and posttest efficacy scores were perfectly correlated. As 
such, there can be no relationship between the collaboration variable and efficacy since there is 
no remaining efficacy variance to predict! In the second set of data, a more realistic and less 
than perfect gain is illustrated (i.e., not all teachers gained the same). Here a part correlation can 
be conducted, but the question remains of what is left of the posttest teacher efficacy after 
removing the influence of pretest teacher efficacy? Is it still efficacy or is it something else? 

The common interpretation is that the remaining variance is uniquely due to the intervention. 
However, it is doubtful that this unique variance is conceptually the same as the originally 
defined efficacy construct. Essentially, covariance corrections of this nature call into question 
the construct validity of the residualized dependent variable. While researchers may begin with 
a posttest measure of teacher efficacy, they may be left with a residualized dependent measure of 
something else. 



INSERT TABLE 5 ABOUT HERE. 
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While there are no rules governing the interpretation of the dependent variable in such 
cases, informed researcher judgment should be invoked to evaluate the constructs that are being 
analyzed. At some point, the dependent variable may not be what we think it is, potentially 
leading to misinterpretation of results. 

Conclusion 

Researchers, especially those doing educational research, are often faced with the 
dilemma of having to use intact groups. Since it is impossible to employ true experimental 
control in these situations, ANCOVA is often employed. By taking out variance attributable to a 
covariate from the dependent variable, ANCOVA "promises" to increase the size of F calculated 
and the possibility of reaching statistical significance. However, problems arise when using 
ANCOVA since the use of intact groups almost guarantees that assumptions necessary for the 
successful use of ANCOVA will not be met. Only after these assumptions are met is it safe to 
use ANCOVA. Part and partial correlation also invoke statistical correction procedures. While 
the assumptions required for part and partial correlation are not identical to those of ANCOVA, 
there are conceptual difficulties in interpreting the residualized dependent variable in most cases. 
After removing variance attributable to the covariate, it is important that the remaining variance 
is comprehensible and maintains appropriate construct validity. 
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Table 1 

Three Summary Tables for Three-level One-Way Design (n=30) 



(a) ANOV A— treatment Ho not rejected 



Source 


SOS 


Df 


MS 


Fcalc 


Fcrit 


Effect 


Decision 


Treatment 

Error 


30 

170 


2 

30 


15.0 

5.6667 


2.6470 


3.32 


15.0% 


Not Rej 


Total (Y) 


200 


32 













(b) ANCOVA with smaller correlation between X and Y and zero correlation between X and 



treatment- 

Source 


treatment Ho not rejected 

SOS Df MS 


Fcalc 


Fcrit 


Effect 


Decision 


Cov. (X) 


30 


1 


30.0 


6.2143 


4.18 


15.0% 


Reject 


Treatment 


30 


2 


15.0 


3.1071 


3.33 


15.0% 


Not Rej 


Error 


140 


29 


4.8276 










Total (Y) 


200 


32 













(c) ANCOVA with higher correlation between X and Y and zero correlation between X and 



treatment- 


-treatment Ho rejected 












source 


SOS 


Df 


MS 


Fcalc 


Fcrit 


Effect 


Decision 


Cov. (X) 


45 


1 


45.0 


10.4401 


4.18 


22.5% 


Reject 


Treatment 


30 


2 


15.0 


3.4800 


3.33 


15.0% 


Reject 


Error 


125 


29 


4.3103 








Total (Y) 


200 


32 
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Table 2 

Summary Tables Illustrating the Necessity for the Covariate and the Treatment to be 
Uncorrelated 



Source 


SOS 


df 


MS 


Fcalc 


Fcrit 


Effect 


Covariate 


45 


1 


45.00 


10.44 


4.18 


22.5% 


Treatment 


30 


2 


15.00 


3.48 


3.33 


15.0% 


Error 


125 


29 


4.31 








Total (Y) 


200 


32 










(a) Treatment and co variate (X) uncorrelated 









Source 


SOS 


df 


MS 


Fcalc 


Fcrit 


Effect 


Co variate 


45 


1 


45.00 


9.32 


4.18 


22.5% 


Treatment 


15 


2 


7.50 


1.55 


3.33 


7.5% 


Error 


140 


29 


4.83 








Total (Y) 


200 


32 










(b) Treatment and co variate (X) partially correlated 







Somce 


SOS 


df 


MS 


Fcalc 


Fcrit 


Effect 


Covariate 


45 


1 


45.00 


8.43 


4.18 


22.5% 


Treatment 


0 


2 


0.00 


— 


3.33 


0.0% 


Error 


155 


29 


5.34 








Total (Y) 


200 


32 










(c) Covariate subsumes Treatment SOS 
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Table 3 

Heviristic Data for Treatment and Control Students on Reading 

(READ) and Achievement (ACHIEVE) Measures (n = 12) 
Student READ ACHIEVE 



Special Education Students Receiving Intervention 


Jennifer 


30 


34 


Suzanne 


30 


36 


Greg 


40 


46 


Kyle 


40 


46 


Natascha 


45 


49 


Alfred 


45 


50 


M 


38.33 


43.50 


SD 


6.83 


6.80 


Students in 


Regular Education Without Intervention 


Stephanie 


50 


60 


Bob 


50 


68 


Timothy 


55 


67 


Micah 


55 


70 


Elizabeth 


65 


80 


Leah 


65 


85 


M 


56.67 


72.67 


SD 


6.83 


9.18 



Note . From ANCOVA with intact groups: Don't do it! by Robin K. Henson. Paper presented at 
the annual meeting of the Mid-South Educational Research Association, New Orleans, LA, 
November, 1998. Reprinted with permission. 
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'Table 4 

ANCOVA Summary Table (n = 12) 



Source 


SOS 


df 


MS 


F 


Effect 


Covariate 


2912.038 


1 


2912.038 


397.383 


96.01% 


Group 


54.926 


1 


54.926 


7.495 


45.44% 


Residual 


65.952 


9 


7.328 






Total 


3032.917 


11 









Note: The Covariate effect size calculated was r2 = .9601 or 96.01%. The Group effect size 
calculated was eta2 = .4544 or 45.44%. The eta2 result was found by dividing the Group SOS 
by the total SOS after the covariate effect was removed (54.926/(54.926 + 65.952) = .4544). In 
ANCOVA, the treatment effects are found in relation to the error scores that are created after 
regressing the covariate (READ) on the dependent variable (ACHIEVE). As such, the Group 
SOS is not divided by the Total SOS but rather by the Residual SOS after removing the 
covariate. 

From ANCOVA with intact groups: Don't do it! by Robin K. Henson. Paper presented at the 
annual meeting of the Mid-South Educational Research Association, New Orleans, LA, 
November, 1998. Reprinted with permission. 
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Table 5 

Hetiristic Data for Pre and Post Teacher Efficacy as Predicted by Teacher Collaboration. 



Teacher Efficacy 



Subject/ 



Statistic 


Pretest 


Posttest 


Teacher Collaboration 


Gerri 


2 


Example 1 
3 


3 


Joy 


2 


3 


4 


John 


3 


4 


2 


Carolyn 


3 


4 


4 


Shawna 


4 


5 


4 


r pretest x posttest ~ 

Residualized sum of squares 

Part r collaboration x residualized posttest 


1.00 

0.00 

= Cannot be computed since the residualized 


Gerri 


2 


Posttest scores are all 0 (constant). 
Example 2 

2 3 


Joy 


2 


3 


4 


John 


3 


3 


2 


Carolyn 


3 


4 


4 


Shawna 


4 


5 


4 


r pretest x posttest 

Residualized sum of squares 

Part r collaboration x residualized posttest 


.981 

1.071 

.926 





Note. All scores were on a 5-point Likert-type scale. 
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Slopes of Subgroups and Total Sample 




GROUP 

° Reg. Ed. Group 
b= 1.257 

° Spec. Ed. Group 
b = .986 

Total Population 
b= 1.405 



READ 



Figure 1 . Slopes of Subgroups and Total Sample 

Note . From ANCOVA with intact groups: Don't do it! by Robin K. Henson. Paper presented at 
the annual meeting of the Mid-South Educational Research Association, New Orleans, LA, 
November, 1998. Reprinted with permission. 
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Appendix 



TITLE ‘ANCOVA heuristic analysis’. 

COMMENT ‘n=12 see Table 1’. 

GET FILE= ‘a:ancova.sav’. 

SET BLANKS=SYSMIS UNDEFINED=WARN PRINTBACK LISTING> 

TEMPORARY. 

SELECT IF (group=l). 

DESCRIPTIVES 

VARIABLES-achieve read 
/FORMAT=LABELS NOINDEX 
/STATISTICS=MEAN STDDEV MIN MAX 
/SORT=MEAN (A). 

TEMPORARY. 

SELECT IF (group=2). 

DESCRIPTIVES 

VARIABLES=achieve read 
/FORMAT=LABELS NOINDEX 
/STATISTICS=MEAN STDDEV MIN MAX 
/SORT=MEAN (A). 

COMMENT ‘determine pooled regression equation’. 

REGRESSION 

/MISSING LISTWISE 
/STATISTICS COEFF OUTS R ANOVA 
/CRITERIA=PIN(.05) POUT(. 1 0) 

/NOORIGIN 
/DEPENDENT achieve 
/METHOD=ENTER read. 

COMMENT ‘compute yhat using weights from all students’. 

COMPUTE yhat = -9.158192 + (1.405085 * read). 

EXECUTE. 



Note . From ANCOVA with intact groups: Don't do it! by Robin K. Henson. Paper presented at 
the annual meeting of the Mid-South Educational Research Association, New Orleans, LA, 
November, 1998. Reprinted with permission. 
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